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DETAILED ACTION 
Information Disclosure Statement 

1. The information disclosure statement (IDS) submitted on September 20, 2004, has been 
considered. 



Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

3. Claims 1-46 are rejected under 35 U.S.C. 102(b) as being anticipated by Riis et al, 
"Multilingual Text-to-Phoneme Mapping," Proceedings of the Eurospeech 2001 Conference. 

4. Regarding claim 1, Riis discloses accepting text spellings (Figure 2, "text input") of 
training words in a plurality of sets of training words, each set corresponding to a different one of 
a plurality of languages (section 3, Multilingual TTP Mapping); for each of the sets of training 
words in the plurality, receiving pronunciations for the training words in the set, the 
pronunciations being characteristic of native speakers of the language of the set, the 
pronunciations also being in terms of subword units at least some of which are common to two 
or more of the languages (Figure 3); and training a single pronunciation estimator using data 
comprising the text spellings and the pronunciations of the training words (Figure 2, 
"Multilingual Recognizer"). 
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Regarding claim 2, Riis discloses (Section 3, Multilingual TTP Mapping) accepting a 
plurality of sets of utterances, each set corresponding to a different one of the plurality of 
languages, the utterances in each set being spoken by the native speakers of the language of each 
set; and training a set of acoustic models for the subword units using the accepted sets of 
utterances and pronunciations estimated by the single pronunciation estimator from text 
representations of the training utterances. 

Regarding claim 3, Riis discloses a first training word in a first set in the plurality 
corresponds to a first language and a second training word in a second set corresponds to a 
second language, the first and second training words having identical text spellings, the received 
pronunciations for the first and second training words being different (Figure 3). 

Regarding claim 4, Riis discloses utterances of the first and the second training words are 
used to train a common subset of subword units (Figure 3). 

Regarding claim 5, Riis discloses (Section 4.1, TTP Mapping) the single pronunciation 
estimator uses a decision tree to map letters of the text spellings to pronunciation subword units. 

Regarding claim 6, Riis discloses training the single pronunciation estimator further 
comprises: forming, from sequences of letters of each training word's textual spelling and the 
corresponding grouping of subword units of the pronunciation, a letter to subword mapping for 
each training word (Section 4.1, TTP Mapping, 4.3, Multilingual acoustic module); and training 
the single pronunciation estimator using the letter-to-subword mappings (section 4). 

Regarding claim 7, Riis discloses training the single pronunciation estimator and training 
the acoustic models is executed by a nonportable programmable device (section 4). 
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Regarding claim 8, Riis discloses generating, for each word in a list of words to be 
recognized, an acoustic word model, the generating comprising generating a grouping of 
subword units representing a pronunciation of the word to be recognized using the single 
pronunciation estimator (section 4.3). 

Regarding claim 9, Riis discloses grouping of subword units is a linear sequence of 
subword units (section 3, Multilingual TTP Mapping to section 4.3, Multilingual acoustic 
module). 

Regarding claim 10, Riis discloses grouping of the acoustic subword models is a linear 
sequence of acoustic subword models (section 4). 

Regarding claim 11, Riis discloses the subword units are phonemes (section 3, 
Multilingual TTP Mapping to section 4.3, Multilingual acoustic module). 

Regarding claim 12, Riis discloses grouping of subwords is a network, and the network 
represents two pronunciations of a word, the two pronunciations being representative of 
utterances of native speakers of two languages (section 4.3). 

Regarding claim 13, Riis discloses processing an utterance; and scoring matches between 
the processed utterance and the acoustic word models (figure 2). 

Regarding claim 14, Riis discloses generating the acoustic word model, processing the 
utterance, and scoring matches is executed by a portable programmable device (section 1, 
Introduction). 

Regarding claim 15, Riis discloses the portable programmable device is a cellphone 
(section \^ Introduction). 
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Regarding claim 16, Riis discloses the utterance is spoken by a native speaker of one of 
the plurality of languages (figure 2; section 4). 

Regarding claim 17, Riis discloses the utterance is spoken by a native speaker of a 
language other than the plurality of languages, the language having similar sounds and similar 
letter to sounds rules as a language from the plurality of languages Figure 2; section 4). 

Regarding claim 18, Riis discloses a method for recognizing words spoken by native 
speakers of multiple languages, the method comprising: generating a set of estimated 
pronunciations, using a single pronunciation estimator, from text spellings of a set of acoustic 
training words, each pronunciation comprising a grouping of subword units, the set of acoustic 
training words comprising at least a first word and a second word, the first and second words 
having identical text spelling, the first word having a pronunciation based on utterances of native 
speakers of a first language, the second word having a pronunciation based on utterances of 
native speakers of a second language (Figure 2; Figure 3; section 3, Multilingual TTP continuing 
to section 4.3, Multilingual acoustic module); mapping sequences of sound associated with 
utterances of each of the acoustic training words against the estimated pronunciation associated 
with each of the acoustic training words (Figure 2; Figure 3; section 3, Multilingual TTP 
continuing to section 4.3, Multilingual acoustic module); and using the mapping of sequences of 
sound to estimated pronunciations to generate acoustic subword models for the subword units in 
the grouping of subwords, the acoustic subword model comprising a sound model and a subword 
unit (Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 4.3, Multilingual 
acoustic module). 
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Regarding claim 19, Riis discloses a method for multilingual speech recognition 

comprising: accepting a recognition vocabulary that includes words from multiple languages 

< 

(Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 4.3, Multilingual acoustic 
module); determining a pronunciation of each of the words in the recognition vocabulary using a 
pronunciation estimator that is common to the multiple languages (Figure 2; Figure 3; section 3, 
Multilingual TTP continuing to section 4.3, Multilingual acoustic module); and configuring a 
speech recognizer using the determined pronunciations of the words in the recognition 
vocabulary (Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 4.3, 
Multilingual acoustic module). 

Regarding claim 20, Riis discloses accepting a training vocabulary that comprises words 
from multiple languages (Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 
4.3, Multilingual acoustic module); determining a pronunciation of each of the words in the 
training vocabulary using the pronunciation estimator that is common to the multiple languages 
(Figure 2; Figure 3; section 3, Multilingual TTP continuing to section 4.3, Multilingual acoustic 
module); configuring the speech recognizer using parameters estimated using the determined 
pronunciations of the words in the training vocabulary; and recognizing utterances using the 
configured speech recognizer (Figure 2; Figure 3; section 3, Multilingual TTP continuing to 
section 4.3, Multilingual acoustic module). 

Regarding claims 21-46, claims 21-46 are computer program product and/or apparatus 
claims similar in scope and content to the method claims of 1-20, and are therefore rejected 
under similar rationale. 
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Conclusion 

The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Schalkwyk (US Patent No. 7,149,688) 
Riis (US Patent No. 7,043,431) 
Sabourin (US Patent no. 6,912,499). 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Angela A. Armstrong whose telephone number is 571-272-7598. 
The examiner can normally be reached on Monday-Thursday 1 1 :30-8:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick N. Edouard can be reached on 571-272-7603. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 



Application/Control Number: 10/716,027 
Art Unit: 2626 



Page 8 



Information regarding the status of an application may be obtained from the Patent 



Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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